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Project  Summary 

Recent  theoretical  developments  in  the  areas  of  latent  variable  modeling,  random 
coefficient  modeling,  multilevel  modeling,  missing  data  theory,  and  empirical 
Bayes  estimation  provide  interesting  research  opportunities  for  the  psychometric 
modeling  of  the  relationship  between  job  performance  and  test  performance. 
These  developments  suggest  new  approaches  to  the  assessment  of  predictive 
validity  of  tests  as  well  as  to  problems  of  selection  and  assignment.  The  purpose 
of  the  project  is  to  use  these  theories  as  a  basis  for  developing  new  psychometric 
methods  that  have  the  potential  for  better  understanding  the  predictive  validity  of 
tests  like  the  ASVAB  and  improving  the  selection  and  assignment  of  recruits  for 
military  jobs. 

Predicting  job  performance  along  with  the  selection  of  recruits  into  military  jobs 
appear  particularly  important  to  the  military  at  this  point  in  time.  Reflecting  the 
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continuing  demand  from  Congress  for  the  quantification  of  the  relationship 
between  job  performance  and  military  enlistment  standards  (Department  of 
Defense,  1991),  a  DoD  project  is  now  being  completed  (Harris,  McCloy, 
Dempsey,  Roth,  Sackett,  Hedges,  Smith,  and  Hogan,  1991)  which  studies  this 
relationship  using  job  performance  data  from  the  JPM  project  (Green,  Wing 
&Wigdor,  1988;  Wigdor  &  Green,  1991).  In  addition,  service-specific  data  is 
being  collected  on  experimental  tests  and  job  performance  measures  (Department 
of  Defense,  1991,  chapter  6).  This  points  to  the  need  for  basic  research  on 
psychometrics  related  to  tests  and  selection. 

The  research  project  intends  to  develop  new  psychometric  methods  based  on  new 
theoretical  statistical  developments.  These  new  methods  will  be  developed  and 
studied  for  seven  psychometric  topics:  predictive  validity  from  a  latent  variable 
perspective,  job  variation  in  predictive  strength,  issues  of  selective  samples, 
missing  data  issues  related  to  incomplete  performance  measures  and  attrition, 
issues  of  selection  and  assignment  by  latent  variable  models,  prediction  using 
latent  variable  models,  and  subgroup  performance.  The  new  methodology  will 
be  tested  out  on  simulated  data,  matching  real-life  situations  as  well  as  possible. 
Real  data  will  also  be  utilized,  such  as  extensive  data  on  Swedish  military 
conscripts,  as  well  as  the  GATE  employment  test  for  which  job  performance 
measures  are  available  in  a  variety  of  civilian  jobs  (Hartigan  &  Wigdor,  1989). 

It  is  of  great  interest  to  study  data  related  to  the  ASVAB  and  additional  cognitive 
predictors  together  with  job  performance  measures  from  the  JPM  project  and 
related  performance  data. 

The  present  data  request  is  made  to  provide  illustration  of  new  methodology  in 
the  context  of  predictive  validity  and  the  estimation  of  a  job  performance 
equation.  The  interest  in  estimating  a  job  performance  equation  is  in  line  with 
HumRRO's  work  on  a  linkage  model  using  JPM  data.  I  became  familiar  with 
these  efforts  in  my  role  as  member  of  the  National  Academy  of  Sciences/National 
Research  Council  Military  Committee  on  Military  Enlistment  Standards.  A 
primary  interest  of  the  present  project  is  to  illustrate  expansions  of  such  a 
performance  equation  using  structural  equation  modeling  with  a  latent  variable 
formulation  for  the  cognitive  predictor  variables  of  ASVAB  as  well  as  additional 
predictors.  The  latent  variable  model  draws  on  psychometric  work  by 
Gustafsson  (1988,  1989).  Issues  of  selectivity  and  missing  data  due  to  selection 
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into  a  military  job  and  attrition  are  accounted  for  in  this  analysis  using  techniques 
described  in  Muthen,  Kaplan,  and  Hollis  (1987)  and  Muthen  and  Hsu  (in  press). 

A  recent  application  of  this  kind  of  methodology  using  Swedish  military  data  is 
reported  in  Muthen,  Hsu,  Carlstedt,  and  Mardberg  (1993). 

Data  Request 

This  request  for  access  to  data  pertains  to  the  Project  A  Concurrent  Validity 
(sample  1 ;  CVI)  data  for  each  of  the  nine  Batch  A  MOS  jobs  (the  jobs  should  be 
identifiable).  In  HumRRO's  work  on  the  Linkage  Project,  the  key  variables  used 
as  predictors  were  the  10  ASVAB  scores,  information  about  high  school 
graduation  (including  GED  information),  and  time  in  service  (which  is  restricted 
for  the  Army  job  sample).  As  criterion,  the  JPM  hands-on  job  performance  test 
score  was  used  (percent  correct).  These  same  data  are  requested  for  the  nine 
Batch  A  jobs,  but  also  the  scores  on  the  other  predictor  and  criterion  variables 
given  in  Campbell  and  Zook  (1991)  Table  4-3  (page  102)  and  Table  4-4  (page 
103).  Furthermore,  data  are  requested  on  the  five  criterion  factors  scores 
described  on  pages  135-136  (Core  Technical  Proficiency,  General  Soldiering 
Prof.,  Effort/Leadership,  Personal  Discipline,  Physical  Fitness  and  Militiary 
Bearing).  Further  variables  of  interest  are  race,  gender,  and  variables  from  the 
PULHES  profile.  SSN  information  is  not  needed.  It  should  be  noted  that 
separate  scores  for  the  subtests  WK  and  PC  are  needed,  not  merely  their  sum. 
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May  17,  1993 


Dr.  Jane  Arabian 
OASD  (FM&P) 

Room  2B271;  The  Pentagon 
Washington  D  C  20301-4000 


Dear  Jane, 


DTIC 

electe 

NOV  17  1993 


As  you  know,  I  have  been  a  member  of  the  NRC  Committe  on  Military 
Enlistment  Standards.  I  just  communicated  with  Anne  Mavor  at  NRC  who 
suggested  that  I  contact  you  about  a  data  access  request  that  I  have.  I 
recently  obtained  a  grant  from  ONR’s  Manpower,  Personnel  &  Training 
R  &  D  Program  (and,  I  think,  co-sponsored  by  the  Office  of  the  Secretary  of 
Defense).  The  topic  is  "Psychometric  Developments  Related  to  Tests  and 
Selection,"  with  a  focus  on  predictive  validity.  In  this  grant  I  state  that  it  is 
desirable  to  try  out  new  methods  on  U.S.  military  data  such  as  the  job 
performance  data  that  HumRRO  has  been  using  to  estimate  their 
performance  equation.  Anne  talked  to  Dickie  Harris  at  HumRRO  who  has 
the  data  but  needs  an  ok  from  the  Services  to  release  it.  She  is  under  the 
impression  that  you  can  ask  for  this  ok.  I  appreciate  your  help  and  I  think 
access  to  the  data  has  the  potential  of  greatly  enhancing  both  my  project  and 
the  estimation  of  the  performance  equation  which  now  is  used  in  modeling 
cost  performance  trade-offs.  I  am  trying  to  get  hold  of  NPRDC  people  (John 
Wolfe)  about  access  to  ECAT  data,  but  if  you  can  facilitate  this  process, 
please  let  me  know.  Enlosed  is  a  brief  description  of  relevant  parts  of  my 
research  project. 


Sincerely, 


'  This  docuraem  has  iaeen  oppioved 
I  for  public  leiecsa  and  sole;  its 
j  distributioD  is  unUaited.  _ 


Bengt  Muthen 
Ph.  D.,  Statistics 
Professor 
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August  12,  1993 


Mr.  Ron  Patsy 

Headquarters  Department  of  the  Army 
Enlisted  Accession  Division 
Classification  and  Budget  Branch 
DAPE-MPA-CB 
Pentagon  Building 
Room  2B  729 

Washington  D.C.  20310-0300 


Dear  Mr.  Patsy, 

Mike  Rumsey  informed  me  that  you  would  be  the  appropriate  person  to 
contact  regarding  the  enclosed  request  for  Project  A  data  access.  The 
context  for  this  request  is  as  follows.  I  recently  obtained  a  grant  from  ONR's 
Manpower,  Personnel  &  Training  R  &  D  Program  (and,  I  think,  co¬ 
sponsored  by  the  Office  of  the  Secretary  of  Defense).  The  topic  is 
"Psychometric  Developments  Related  to  Tests  and  Selection,"  with  a  focus 
on  predictive  validity.  In  this  grant  I  state  that  it  is  desirable  to  try  out  new 
methods  on  U.S.  military  data  such  as  the  hands-on  job  performance  data 
that  HumRRO  has  recently  been  using  to  estimate  the  performance  equation 
in  their  Linkage  project.  I  have  learned  about  this  work  as  a  member  of  the 
National  Research  Council  Committe  on  Military  Enlistment  Standards.  I 
have  communicated  with  Anne  Mavor  at  NRC  and  with  Jane  Arabian  about 
requesting  data  to  illustrate  methods  to  be  developed  in  my  project.  Jane 
Arabian  brought  up  this  issue  in  the  July  meeting  of  the  Manpower 
Accession  Policy  Working  Group  and  suggested  that  I  contact 
representatives  for  the  services. 

Enclosed  is  a  brief  description  of  relevant  parts  of  my  research  project  and  a 
specification  of  the  data  I  would  like  to  use.  In  a  separate  mailing  I  am 
sending  a  recent  paper  illustrating  the  methods  I  have  in  mind  (Muthen,  Hsu, 
Carlstedt,  Mardberg,  1993). 


It  would  be  of  great  use  for  my  project  if  you  could  have  these  data  made 
available  to  me  as  soon  as  possible. 


Sincerely, 


Bengt  Muthen 
Ph.  D.,  Statistics 
Professor 


Phone  number:  (310)390-8587. 
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